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ABSTRACT 

An  algorithm  is  presented  which  finds  the  best-fitting 
pair  of  constants,  in  the  least  squares  sense, 
scalar  data;  we  call  this  pair  of  constants  the 
of  the  data.  The  relationship  of  the  bimean  clustering  to 
the  ISODATA  clustering  algorithm,  and  its  application  to 
image  thresholding,  are  also  discussed. 
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1 .  Introduction 


Let  us  first  consider  the  following  minimization  problem: 

n  2 

minimize  f  *  £  (x.-y)z  (1) 

i=l  1 

where  we  may  assume  without  loss  of  generality  that  x^XjS...^ 
xR.  It  is  well  known  that  f  is  minimized  when  y  is  the  aver¬ 
age  of  the  x^,  that  is, 

1  n 

y  =  ^  2  x.  (2) 

n  i=l  1 

We  now  extend  this  to  the  following  minimization  problem: 

n  2  2 

minimize  f  *  £(min((x.-y)  ,  (x.-v)  )  (3) 

i=l  1  1 

That  is  to  say,  we  are  interested  in  the  best  fitting  pair  of 
constants  to  the  set  x1,...,xn  of  data.  We  refer  to  this  pair 
of  constants  (y,v)  as  the  bimean.  This  is  because,  as  we  shall 


k  5  yk 


v '  <“-*>  i-LlXi  '  ^ 

for  some  k,  lsksn.  That  is,  the  constants  y  and  v  are  such 
that  they  are  averages  or  means  of  subsets  of  the  n  data  points 
We  are  interested  in  the  bimean  because  it'  defines  a 


natural  clustering  of  the  x's  into  two  subsets.  For  example , 
if  the  x's  are  the  gray  levels  of  pixels  in  an  image,  cluster¬ 
ing  can  be  used  to  segment  the  image,  e.g.,  into  objects  and 
background.  Velasco  [1]  recently  showed  that  the  segmentation 
can  be  done  with  the  ISOOATA  clustering  algorithm.  We  will 


show  that  the  ISODATA  algorithm  should  ideally  converge  to 
the  bimean  clustering  result,  but  that  there  are  cases 
where  even  this  ideal  clustering  approach  does  not  select 
the  appropriate  threshold. 

This  minimization  problem  has  attracted  much  recent 
interest.  Hartigan  and  Wong  [2]  present  an  algorithm  for 
k-mean  clustering  which  produces  a  global  minimum  only  for  the 
two-mean  case.  Pollard  [3]  discusses  the  convergence  of 
the  k-mean  clustering.  We  consider  this  question  briefly  in 
Section  4  and  show  that  the  two-class  case  of  the  ISODATA 
clustering  algorithm  is  relevant  to  the  bimean.  Fisher 
[4]  discusses  algorithms  for  clustering,  but  does  not  con¬ 
sider  the  special  case  of  two  means. 

In  Section  2,  we  develop  the  bimean  clustering  algorithm, 
and  in  Section  3,  we  show  how  the  algorithm  is  applied  to 
image  segmentation.  Concluding  comments  are  presented  in 
Section  4 . 
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2.  The  Bimean  Clustering  Algorithm 

In  this  section  we  present  an  algorithm  to  compute  the 
values  of  y  and  v  such  that 

f  ■  2  min ((x.-y)2,  (x.-v)2)  (5) 

i-1  1  x 

is  minimized  where  x, <x0< . . . <x„ .  First  note  that  since  f 

l—  2—  —  n 

is  continuous  and  fj^O,  a  minimum  exists.  We  shall  show  that 
if  among  the  n>l  values  x^,  ...,xn  there  are  at  least  two  dis¬ 
tinct  values^  the  minimum  is  attained  for  some  y<v. 

k  2 

Theorem  It  Let  f  ( ]a )  =  Z  (x.-y)  where  x.sx  <...<x.  .  Then 

i=l  1  z  K 

there  exists  one  value  of  y  at  which  the  minimum  is  attained. 


given  by 


and  for  y^y*,  f(y)>f(y*). 


y*  -  ^  2 
K  i-1  1 


Proof :  With  y*  as  defined  above,  we  may  write 

k  2 

f (y)  -  2  <x.-yr 

i-1  1 


f (y)  -  2  (x. -y*+y*-y) 2 

i-1  1 


which  can  be  expanded  as 

k  „  k 


f(y)  -  Z  (x.-y*)2  +  Z  (y-y*)2  +  2 (y-y*)  Z  (y*-x, ) 
i-1  1  i-1  i-1  1 


2  (x.-y*)2  +  k(y-y*) 
i-1  1 


since  the  last  term  is  zero  with  y*  as  defined  above.  The  mini¬ 
mum  is  obtained  when  y-y*,  and  for  y^y*,  f(y)>f(y*).  || 


We  now  consider  the  function 


f(y,v)  =  Z  min((x.-y)  ,  (x.-v)  )  (10 

i-1  1  1 

As  a  corollary  to  Theorem  1,  we  now  show  that  the  minimum  of 
f  ( y ,  v)  cannot  occur  for  y*  v. 


Theorem  2:  If  the  n>l  values  x15x^5...<x_  contain  at  least 
— — — —  i  2  n 

two  distinct  values,  the  minimum  is  attained  for  some  y<v. 


Proofs 
for  y—  v. 


We  assume  to  the  contrary  that  the  minimum  is  attained 


Theorem  1  implies  a  minimum  at 

n 


"  =  »  Ji** 


(11 


Therefore  y<xn»  because  x^,...,xn  contain  at  least  two  dis 

tinct  values.  We  now  consider 

—  n  —  2  2 
f(u,xn)  *  Z  min((xi-y)  , (xi-xn)  ) 


(12 


and  we  see  that 


«  n-1 

f(U/X  )  5  Z  (x.-y) 
n  i-1  x 


2 


and  also  that 


n-1 

Z 

i-1 


(x^y)2  < 


n  _ 
Z  (x . -y) 
i-1  1 


since  x  -y>0. 
n 

_  _  n  —  2 

Since  f(y,y)  *  Z  (x.-y)  and  from  Theorem  1  follows  that 
i-1  1 

this  expression  attains  the  unique  minimum  at  (y,y). 

We  see  that  f(y,x  )<f(y,y) 

which  is  a  contradiction  to  our  assumption  that  f(y,v)  is 


minimized  when  y=v.  We  conclude  that  v  if  the  n>l  values 


x1<x2<...<xn  contain  at  least  two  distinct  values.  || 

From  the  very  definition,  it  follows  that  f  (y,  v)»f (v,  y) . 
With  this  fact  and  Theorem  2,  we  can  restrict  the  domain  of 
definition  of  the  function  f(y,v)  to  be  {(y,v):  y,v€IR  and 
y<v>. 


We  now  define  the  numbers  yk  and  vk  to  be 

1  ^ 
uk  “  k 

1  n 

vv  “  riv  Z  X, 

k  n  k  i-k+1  i 


(13] 


for  all  k,  I5k5n.  We  prove  our  final  theorem  from  which  the 
bimean  clustering  algorithm  follows. 

Theorem  3:  If  (y*,v*)  is  the  minimum  of  f  (y,v)  for  y<v  then 
there  exists  an  index  k  such  that 


u*  -  Ukr  v*  ■  Vk 
xkSj(y*+v*) 

xk+l>I(y*+v*)  (14) 

Proof:  Let  k  be  the  largest  index  such  that 

xk  <  j(y*+v*) 

is  true.  We  can  write  f(y*,v*)as 

k  .  n  ~ 

f  (y*,v*)  —  Z  (x.-y*P  +  Z  (x,-v*r 
i-1  1  i-k+1  1 

Assume  to  the  contrary  that  y*j<yk  or 

Case  1:  y*^yk« 

n  22 

Consider  f(yk,v*)  -  ^I^min(  (Xj-yk)  ,  (x^-v*)  ) 


(15) 


< 


( 


z 

i-1 


(xi-yk) 


£ 


+ 


z  (X.-V*)‘ 

i=k+l  1 


By  Theorem  1,  the  first  term  of  equation  (16)  is  minimized, 
thus 

k  5  n  „ 

f<y*,v*)  >  Z  (x.-y.r  +  I  (x.-v*r  C 

i=l  1  K  i=k+l  1 


From  equations  (16)  and  (17) 

f(Uk/V*)  <  f(y*,v*) 

and  we  have  achieved  a  contradiction  to  the  assumption  that 
the  minimum  of  f(y,v)  is  attained  at  (y*,v*). 


Case  2:  v*^vk. 

The  proof  for  this  case  is  analogous  to  that  for  the  first 
case  and  shall  be  omitted.  Thus  we  have  shown  that  lJ*sUk 
and  for  some  k,  l<ksn.  )i 

With  Theorem  3  in  hand,  we  present  the  bimean  clustering 
algorithm: 

Step  1)  Find  the  set  K  of  indices  which  satisfy 

xk  5  ^*yk+vk* 

Xk+1  >  i(W 

Step  2)  For  all  k€K  evaluate  f(l*k'vk) 

Step  3)  Find  the  minimum  of  ^(^k'vk)  for  all  k€K.  Set 
and  v*»v^  where  j  is  the  largest  element 
of  K  for  which  the  minimum  is  attained. 


3.  Application  to  thresholdin 


A  possible  application  of  the  bimean  is  to  segment  an 
image  so  as  to  separate  objects  from  their  background,  by 
clustering  the  gray  levels  of  the  image's  histogram  into 
two  clusters.  Thus,  the  index  k  in  the  bimean  clustering 
algorithm  is  the  gray  level  above  which  the  gray  levels  be¬ 
long  to  the  objects,  and  the  cluster  of  gray  levels  below 
the  index  k  belong  to  the  background. 

Figure  1  shows  results  of  applying  the  bimean  clustering 
algorithm  to  a  set  of  infrared  images  of  tanks.  The  original 
images  are  shown  in  the  first  column,  and  the  bimean  results 
in  the  third  column.  The  second  column  shows  ISODATA  results 
(see  below) ,  with  the  initial  threshold  taken  at  the  mean 
gray  level  of  the  image.  We  see  that  the  first  five  images  are 
reasonably  segmented  by  the  bimean  algorithm,  but  the  last 
three  are  not;  and  the  ISODATA  results  are  not  as  good  (e.g., 
the  fourth  image  is  poorly  segmented) . 

Velasco  [1]  showed  that  a  two-class,  one-dimensional  ISODATA 
clustering  algorithm  (e.g.,  [5])  could  be  used  to  segment  images 
into  two  gray  level  classes.  Our  experiments  show  that  this 
is  not  always  the  case.  ISODATA  is  an  iterative  process  based 
on  the  same  distance  measure  that  we  are  minimizing  in  the  bimean 
algorithm;  but  ISODATA  may  converge  only  to  a  local  minimum  of 
this  measure,  whereas  our  algorithm  finds  the  global  minimum. 

In  spite  of  this,  we  do  not  always  obtain  good  thresholds,  and 
neither  does  ISODATA. 


4.  Discussion  and  concluding  remarks 


We  have  presented  a  new  algorithm  for  clustering  single 
dimensional  data  into  two  clusters.  The  computation  is 
relatively  quick,  and  the  equations  can  be  rewritten  so  as 
to  only  perform  a  single  pass  through  the  data. 

The  ISODATA  algorithm  should  ideally  converge  to  the  Bimean 
but  this  requires  a  suitable  initial  choice  of  means  for 
ISODATA.  For  example,  if  the  second  mean  is  set  equal  to 
one  outlier,  the  ISODATA  algorithm  converges  but  possibly  not 
to  the  true  bimean.  Thus,  the  Bimean  algorithm  is  a  more 
reliable  method  of  obtaining  a  globally  optimal  threshold  than 
iterative  algorithms  such  as  ISODATA.  Figure  1  shows,  however, 
that  this  method  does  not  always  perform  well  in  practice. 
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